Papers with projection layer
pNLP-Mixer: an Efficient all-MLP Architecture for Language (2023.acl-industry)
Copied to clipboard
| Challenge: | large pre-trained language models are impractical for on-device applications due to their size and inference cost. |
| Approach: | They propose an embedding-free MLP-Mixer model for on-device NLP that achieves high weight-efficiency thanks to a novel projection layer. |
| Outcome: | The proposed model beats state-of-the-art of tiny models by 97.8% on two datasets . it beats mBERT on MTOP and multiATIS, while using 170x less parameters . |
Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability. |
| Approach: | They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning. |
| Outcome: | The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models. |
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response (2024.findings-naacl)
Copied to clipboard
Zihao Deng, Yinghao Ma, Yudong Liu, Rongchen Guo, Ge Zhang, Wenhu Chen, Wenhao Huang, Emmanouil Benetos
| Challenge: | Large Language Models have shown immense potential in multimodal applications, but convergence between textual and musical domains remains unexplored. |
| Approach: | They propose a system that aligns music representations with a frozen LLM . they train the system on an extensive music caption dataset and fine-tune it with instructional data . |
| Outcome: | The proposed system bridges the gap between music audio and textual contexts by combining music captions with a frozen model . it performs well in generating music caption and composing music-related Q&A pairs . the proposed system is available for free download at http://www.musilingo.com/ . |